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Abstract 

We consider a discrete-time d-dimensional process {X n } on with a background process 
{</„} on a finite set, where the front process {X n } is skip free in all directions. We assume 
that the joint process {3^n} = {(X n , J n )} is Markovian and that the transition probabilities of 
the front process {X n } are modulated depending on the state of the background process { J n }. 
This modulation is space homogeneous in the inside of Z+ and in each boundary face of Z+. 
We call this model a d-dimensional skip-free reflecting random walk with a background process 
and obtain sufficient conditions on which it is positive recurrent and on which it is transient. 
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1 Introduction 

We consider a discrete-time (i-dimensional process {X n } = {{X\^ n , A2 jn , ^d,n)} on Z+ with a 
supplemental process { J n } on a finite set, where Z + is the set of nonnegative integers. We call the 
supplemental process a background process. The front process {X n } is skip free in all directions, 
which means that the increment of each element of the front process takes values in {—1, 0, 1}. We 
assume that the joint process {i^n} = {(X n , J n )} is Markovian and that the transition probabilities 
of the front process {X n } are modulated depending on the state of the background process {J n }. 
This modulation is space homogeneous in the inside of lA, and in each boundary face of "lA. 
We call this model a (i-dimensional skip-free reflecting random walk with a background process 
(cZ-dimensional RRW-WBP, in short). Our aim is to exploit sufficient conditions on which the 
d-dimensional RRW-WBP is positive recurrent and on which it is transient. 

The multidimensional RRW-WBP is an extension of the ordinary multidimensional reflecting 
random walk and it has a lot of potential applications such as those of queueing models; Applications 
of queueing models spread over various areas, for example, production systems, information network 
systems and certain service systems, and such a system is often modeled by a multiclass queuing 
network because it consists of some nodes and contains some kinds of customer. Here we mention 
one typical example: A single-class (i-node generalized Jackson network with Markovian arrivals 
and phase-type services. The behavior of this model can be represented as a continuous-time 
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Markov chain {Y(t)} = {(X\(t), ^(t), Xa(t), J(t))}, where Xi(t) is the number of customers 
in the ith node at time t and J(t) is the state (phase) of the process combining the Markovian 
arrival processes and phase-type service processes (see, for example, Neuts [11] and Latouche and 
Rawaswami [7]). This {Y"(i)} is a continuous-time version of d-dimensional RRW-WBP and, by the 
uniformization technique, we obtain a discrete-time d-dimensional RRW-WBP, {l^nji that has the 
same stationary distribution as {Y(t)} if it exists. This implies that {Y(t)} is positive recurrent if 
and only if {l^n} is positive recurrent, and in order to check positive recurrence of the former, it 
suffices to check that of the latter. Furthermore, in the framework of multidimensional RRW-WBP, 
we can add some kinds of service discipline and some kinds of interaction between nodes (queues) 
to the model, by means of Markovian service processes (see, for example, Ozawa |12|). Therefore, 
understanding positive recurrence and transience of multidimensional RRWs-WBP provides an 
important basis for analyzing such a queueing model. Of course, stability of queueing networks 
was studied in a lot of literature in recent decades (see Meyn and Down [9j, Dai [[3], Bramson pQ 
and references therein), but in the literature, in order to study the stability, they used stochastic 
models differed from ours such as fluid limit models. 

Recently, asymptotics for the stationary distributions of reflecting random walks were studied 
in some literature (see, for example, Miyazawa |10| and references therein) and those of two- 
dimensional quasi-birth-and-death (QBD) processes, which correspond to two-dimensional RRWs- 
WBP, were studied in Ozawa [13]. Understanding positive recurrence and transience of multidi- 
mensional RRWs-WBP provides also a basis for such studies. 

Multidimensional RRWs without a background process are ordinary multidimensional RRWs, 
and their positive recurrence and transience were studied by Malyshev [8] and Fayolle et al. [5]. 
They took the following approach. Let R+ be the set of nonnegative real numbers and consider 
a d-dimensional RRW, denoted by {X n } = {(Xi >n , X2 >n , •••> ^d,n)}- First, they defined induced 
Markov chains, each of which is, roughly speaking, a kind of projection of onto some boundary 

face (for a precise definition, see Subsection 14. ip and then defined the second vector field on 
on which each point of is assigned the mean increment vector of {X n } evaluated by the 
stationary distribution of a certain induced Markov chain. Considering paths on the second vector 
field and applying Foster's criterion, they gave sufficient conditions on which the RRW is 
positive recurrent and on which it is transient, in terms of certain conditions for the second vector 
filed. In this paper, we take a similar approach, but do not use the concept of second vector filed. 
Instead of doing so, we consider a certain embedded Markov chain of the d-dimensional RRW- 
WBP, {Y n } = {(X n , J n )}, and directly apply Foster's criterion to the embedded Markov chain. 
As a result, we obtain sufficient conditions on which {l^ n } is positive recurrent and on which it 
is transient; each of the sufficient conditions is given by a condition on which there exists at least 
one fisible solution to a system of linear and quadratic inequalities whose coefficients are given by 
the elements of the mean increment vectors of the front process {^n} evaluated by the stationary 
distributions of the induced Markov chains generated from Applying these results to two 

and three-dimensional models, we also obtain the corresponding sufficient conditions in explicit 
form. 

The rest of the paper is organized as follows. In Section [21 the <i-dimensional RRW-WBP we 
consider is described in detail. In Section [3l Foster's criterion and its variations are introduced. In 
Section (4[ sufficient conditions on which the d-dimensional RRW-WBP is positive recurrent and 
on which it is transient are obtained, and in Section \5\ those results are applied to two and three- 
dimensional models. A simple three-queue model with service interactions is presented in Section 
El as an example of three-dimensional RRW-WBP. 
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2 Reflecting random walk with a background process 



Let d be a positive integer and define a set N as N = {1, 2, d}. Define a function y> : — >■ A/" 

as 

<£>(x) = iV \ {Z G iV : = 0} for cc = (xi, x 2 , x d ) G Z+, 

where M is the set of all subsets of TV. Note that we treat vectors such as x as column vectors in 
formulae, except for probability vectors, which we treat as row vectors. For A G Af, let S be a 
finite set, say S A = {1,2, ...,s A } for some positive integer s A . Let a state space S be defined as 

and consider a time-homogeneous Markov chain {l^n} = {(X n , J n )} on the state space S; we call 
the process {X n } = {(Xi n ,X2 n , ■ ■ ■ , X dn )} the front process of {l^n} and {J n } the background 
process of it. From the definition, it can be seen that the transition probabilities of the front 
process are modulated depending on the state of the background process. We call the process 
{Y n } a (i-dimensional reflecting random walk with a background process (d-dimensional RRW- 
WBP, in short) and denote it by C. Throughout the paper, we assume the following condition. 

Assumption 2.1. C is irreducible. 

Furthermore, we assume that individual processes {Xi n }, I G N, are skip free and introduce a 
kind of space-homogeneous structure of transition probabilities, as follows. For A G Af, let B A be 
the boundary face defined as 

B A = {(x,i)eS:<p(x) = A}; 

B N is the interior space of 5, but we also call it a boundary face. Transition probabilities are space- 
homogeneous in each boundary face and they are represented as, for (x,i) G S, z G {—1,0, l} d 
such that x + z G Z% and j G S^ x+Z \ 

P(Y n+l = (x + z,j)\Y n+1 = (x,i))=pt {x) * {x+z \ij). (2.1) 

Let £ n = (£i,n> %2,n> ■•■) £d,n) be the vector of increments defined as £ n = X n — X n _i. By the 
assumption of skip-free property, we have £ n G {—1,0, l} d . For y G S, we denote by a(y) the 
vector of the conditional mean increments given that the chain is in state y, i.e., 

&(y) = (a 1 (y),a 2 (y),--- ,a d (y)), a t (y) = E(^ n+1 \Y n = y), I G N. 

3 Preliminaries 

We present some existing results which are used for proving theorems after. Note that, in this 
section, notations are defined independently of other sections. 

Let {Z n } be a discrete-time Markov chain on a countable set S and assume that it is time- 
homogenous and irreducible. The following proposition is called Foster's criterion and it gives a 
necessary and sufficient condition on which a Markov chain is positive recurrent. 
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Proposition 3.1 (Foster's criterion). The Markov chain {Z n } is positive recurrent if and only if 
there exist a positive number e, a finite subset V C S and a lower bounded real function f(i), i & S, 
such that 

E(f(Z n+1 ) - f(Z n ) \Z n = i)< -e, i€S\V, (3.1) 
E(f(Z n+1 ) | Z n = i) < oo, ieV. (3.2) 

Exclusively divide the state space S into a finite number of nonempty subsets, say Vfc, k = 
1,2, ...,ms, and define a function r(i) as 

t(i) = T k if i G 14, 

where Tf., k G {1, 2, mj), is a positive integer. Furthermore, define a strictly increasing time 
sequence {t n } as 

t n+ i =t n + r(Z tn ), n > 0, t = 0, 

and consider a stochastic process {^ n } defined by Z n = Z tn , n > 0. Obviously this process {Z n } 
is a kind of embedded Markov chain of {Z n }. If the embedded Markov chain is irreducible, then a 
recurrence time for each state in the embedded Markov chain is always greater than or equal to the 
corresponding recurrence time for the same state in the original Markov chain. This implies that if 
{Zn} is positive recurrent then {Z n } is also positive recurrent, and hence we obtain the following 
corollary, which is a simple version of Theorem 2.2.4 of Fayolle et al. [5]; also see Theorem 3.2 of 
Fayolle [4] and Proposition 4.5 of Bramson pQ. 

Corollary 3.1. Assume that the embedded Markov chain {Z n } is irreducible. If there exist a 
positive number e, a finite subset V C S and a lower bounded real function f(i), i 6 S, such that 

E{f{Z n+1 ) - f(Z n ) \Z n = i)< -e, i G S \ V, (3.3) 
E(f(Z n+1 ) | Z n = i) < oo, ieV, (3.4) 

then the Markov chain {Z n } is positive recurrent. 

In order to prove that a Markov chain is transient, we can use the following proposition, which 
is a simplified version of Theorem 2.2.7 of Fayolle et al. [5]. 

Proposition 3.2. Assume that the embedded Markov chain {Z n } is irreducible. If there exist a 
real function f(i), i G S, and positive numbers e, c and b such that, for A = {i G S : f{i) > c}, 

(i) A + 0, A c ^ 0, 

(ii) E(f(Z n+1 ) - f(Z n ) \ Z n = i)>e, ieA, and 

(Hi) the inequality \f(j) — f(i)\ > b implies P{Z\ = j | Zq = i) = 0, 
then the Markov chain {Z n } is transient. 
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4 Positive recurrence and transience 



In this section, we deal with a d-dimensional RRW with a background process, C = {Y n } = 
{(X n , J n )}, introduced in Section^ and derive sufficient conditions on which C is positive recurrent 
and on which it is transient. For the purpose, we introduce three kinds of Markov chain generated 
from C, including induced Markov chains, and then obtain approximations for the expected time- 
averaged increments of the front process {X n }. This approximation plays an essential role in 
deriving main results. 

4.1 Markov chains generated from C 

Hereafter, we use the following notations. For A C N, we denote by x A a vector in Z^ 4 whose 
elements are indexed in A, i.e., x A = (xi, I G A), and by 2 A the | A | -dimensional vector of 2's whose 
elements are indexed in A, where \A\ is the cardinality of set A. We also denote by (x A ,y N \ A ) the 
combination of x A and y N \ A ; the I th element of (x A ,y N \ A ) is xi if / G A and yi if I G N \ A. For 
example, when d = 5 and A = {1,3}, we have x A = (xi,xs), y N \ A = (1/2,114,1/5) and (x A ,y N \ A ) = 
(xi, U2 , xs, 1/4, 2/5). We also use this notation for expressing a part of a given vector; For x = (xi, I G 
N) and A C N, we denote (xi,l G ^4) by x A and represent x as x = (x A ,x N \ A ) = (x N \ A ,x A ). 

4.1.1 Expanded Markov chain 

We consider a Markov chain obtained from C by removing some boundaries. For A C N such that 
A ^ 0, let Sa be a state space defined as 

Sa = U U ({(^,^)}x^ XA ' 2A )), (4.1) 

X N\A 6 2 d_|j4 ' x A &\ A \ 

and consider a skip- free partially-reflecting random walk with a background process on 5,4, denoted 
by {(X n , J n )} = {((X^ A , X A ), J n )}, whose transition probabilities are given by, for (x,i) = 
((x N \ A ,x A ),i) gSa and z = (z N \ A ,z A ) G {-1,0, l} d such that (x + z,j) G Sa, 

P{(X n+1 ,J n+1 ) = (x + z,j)\(X n ,J n ) = (x,i)) = pt (x ' 2 M * +z ' 2 (4.2) 

where pz^ x,>v ^ x+z \i,j) is given in expression (|2.ip . We call {(X, J ra )} an expanded Markov chain 
of C concerning A and denote it by La- 

4.1.2 Induced Markov chain 

For A C N such that A ^ 0, we define a state space S A as 

5 A = (J ({/^JxS^^), (4.3) 

a;Ar\^gZ+" |A| 

and introduce a Markov chain £^ on S , called an induced Markov chain, according to Maly- 
shev [8] and Fayolle et al. |S]; C A is the (d — |A|)-dimensional RRW with a background pro- 
cess, {(X n ,J n )}, whose transition probabilities are given by, for (x N \ A ,i) G S A and z N \ A G 
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{-1,0, l} d -\ A \ such that x N \ A + z N \ A G Z d ~ lAl and for j G ^(^+^,2^ 
p((*£f. J n+1 ) = (^ + z^, J ) | (X^Vn) = (x N \ A ,i)) 

= £ PS^ M " ,?VWVA ^ ) (<,i). (4-4) 

z A G{-l,0,l} |A| 

Throughout the paper, we assume the following condition. 

Assumption 4.1. For all A C N such that A^%, C A is irreducible. 

For A C N such that A ^ 0, if C A is positive recurrent, then we denote by iv A = (n A (x N \ A , i), (x N \ A , i) G 
S A ) its stationary distribution. Note that L N is a finite Markov chain and it is always positive 
recurrent. For C A being positive recurrent, we define the d-dimensional mean increment vector, 
a(A) = (o,(A), I G AO, by 

ai (A)= a l ((x N \ A ,2 A ),i)7r A (x N \ A ,i), leN. (4.5) 

{x N \ A ,i)£S A 

From the definition, it can be seen that ai (A) = for all I G N \ A. 

Remark 4.1. From the definition of induced Markov chain and that of extended Markov chain, 
it can be seen that, for each A C N, A ^ $, the probability law of the induced Markov chain C A 
is identical to that of the partial process (marginal process) {(X^ A , J n )} of the extended Markov 
chain La- In other wards, C A is the projection of La onto the boundary face B N \ A . 

4.1.3 Embedded Markov chain 

For A C N such that 4/8, let Ka be a positive integer and assume that, for A,B C N, if 
\A\ > \B\ then Ka < Kb- We inductively define, for A C N, a subset Va C S as follows (for the 
case of d = 2, see Fig. [[]). 

Step 1 Set k := d and V$ := S. 

Step 2 For A C N such that \A\ = k, set Va ■= V n {(x,i) G S : xi > K A , I G A}. Set 
H) : = H \ (l)AcN,\A\=k V a) and k := k - 1. 

Step 3 If k > then go to Step 2, otherwise we obtain Va for all vl C N. 

It is obvious that \J AcN Va = S and that Va n Vs = for all 4,5 C iV such that 4 / B. 
Thus, {Va ■ A C iV} is a partition of 5. We also see that Vq is finite. For J/G.S, define a function 
r(y) as 

r(y) = T A if y G Va for some A C N, 

where Ta, A C N, are positive integers. We assume that T$ = 1 and that, for yl C N such that 
A ^ 0, Ta < Ka- Furthermore, define a time sequence {t n } as t n+ \ = t n + r(l^ n ), to = 0, and 
a Markov chain {l^ n } as Y n = Y t „, n > 0. We call {Yn} the embedded Markov chain of L 
and denote it by L. Since the original Markov chain L is irreducible, we can make the embedded 
Markov chain L irreducible by appropriately setting parameters Ta, A C N, A ^ 0, and this 
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does not restrict the values of the parameters from above. Hence, hereafter we assume that the 
parameters Ta, A C N A ^ 0, satisfy that condition. By Corollary 13. 1} if there exists a set of 
parameters {(Ka,Ta) ■ A C N, A ^ 0} such that the condition of the corollary holds, then the 
Markov chain C is positive recurrent. 

Remark 4.2. Since the front process of the RRW-WBP {Y n } is skip free in each direction, it can 
be seen that, for A C N such that A ^ 0, ifY^ £ Va then the Markov chain {Yn} does not visit 
any state on the boundary face B N \ A during the time interval [t}-, ifc+l]- Thus, in stochastic sense, 
the process {Y n } behaves just like the expanded Markov chain La during that time interval. 




#{1,2} K {1} Xx 



Figure 1: Partition of the state space (d = 2) 



4.2 Time average of increments 

Here we consider the expected time-averaged increments of the front process and obtain their 

approximations, which will be used as an important tool for deriving our main results. 

Consider a (f-dimensional RRW-WBP {Y n } = {(X n , J n )} on state space S. Let M p be the 
index set of the positive-recurrent induced Markov chains, i.e., 

M p = {A C N : A 7^ 0, C A is positive recurrent}. 

Since C N is always positive recurrent, M p is not empty. For A C N such that A ^ 0, let M A be 
the set of the subsets of N including all elements of A, i.e., M A = {B C N : A C B}, and M A be 
defined as M A = M A PI M p . Note that if C A is positive recurrent, then A G M A . For y S S, define 
the vector of the expected time-averaged increments, g(y), as 



g{y) = (gi(y), i e n), 9l { y ) = £^_J^, in Y = y j,i€N 



where £j n = Xi n — X^ n _i. The following proposition asserts that g(y) can be approximated by 
a linear combination of ol(A), A € M p , the mean increment vectors evaluated by the stationary 
distributions of the positive-recurrent induced Markov chains. 
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Proposition 4.1. For any e > 0, there exist the set of parameters {(Ta,Ka) : A C N, A ^ 0} 
satisfying the conditions mentioned in Subsection \4-l-3\ and substochastic vectors (jja(B), B € 
M A ), A C N, A ^ 0, such that, for every y £ Vq, if y € Va for some A C N, then we have 



gi(y) - E PA{B) ai (B) 



<e,leN, (4.6) 



where some ofpA(B), B € N A , may be zero and Ka can be taken sufficiently largely, independently 
of Ta- Especially, if C A is positive recurrent, we have 

\ 9l (y) - ai {A)\ < e, I e N. (4.7) 

For simplicity, we denote gi(y) having property |^.6|) by 
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Proof. For A <Z N such that A ^ 0, let = {(X n , J n )} be the extended Markov chain La and 

define the conditional expected time-average of increments as, for I € N and y £ <Sa) 

9i(AJ/)=^( 7^E&,n (*0,Jb)=V 



( 1 - 

x 71=1 



where = JQ, n - Xj jn _i. By Remark[01 if V & Va, then we have gi(y) = gi(A,y). Therefore, 
we evaluate g~i(A,y) instead of gi(y). For y G Va, 

1 Ta ^ 

9i{A,y ) = ^2 E E i&,n+i I ( x n, Jn) = y n )P((X n , J„) = y n | (X , J ) = y ) 

= ^- T E E E ^(4n + il((^ A ,^),J n ) = ((^,2^),i n )) 

x p(((X^,^),J n ) = ((^,^),j n ) | (X , Jo) = y 
= f T E E m((x^\2 A ), Jn ) 

A "=° (»^,in)^ 



x P((X^ A , J n ) = (a^V 4 , j n ) | (X , Jo) = y ) 
E 0f- E «*((*n^),J n ) (X ,J ) = y ) 

^ A n=0 ' 



(4.8) 



where y n is expressed as y n = (x n ,j n ) = ((xn ,x A ),j n ). By Remark 14.11 the probability law of 
the partial process {(X^ A ,J n )} is identical to that of the induced Markov chain C A . Thus, by 
the ergodic theorem for Markov chains, if C A is positive recurrent, then for a sufficiently large Ta, 
we obtain 

\gi(A,y )-ai(A)\ < e, I e N, 
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where ai (A) is given by expression f|4.5j) : this is identical to inequality (|4.6p and Ka can be taken 
arbitrarily largely. 

Next, we prove the proposition in the case where a concerning induced Markov chain is not 
positive recurrent, by induction. Consider the induced Markov chain C N , where \N\ = d. Since 
C N is positive recurrent, inequality (|4.6p holds for some T/v > and Kjq can be taken arbitrarily 
largely. Suppose that there exists some parameter set {(Ta,Ka) '■ A C N, \A\ > k} such that, for 
all A C N, \A\ > k, inequality P~6|) holds. Let A be a subset of N satisfying \A\ = k - 1. If the 
induced Markov chain C A is positive recurrent, then inequality (14.61) holds for some Ta > and 
Ka can be taken arbitrarily largely. If it is not, we introduce an embedded Markov chain generated 
from the expanded Markov chain La, denoted by {i^} = {{X-\, Jn)}. For B G M A , let Vb be a 
subset of 5,4, obtained by expanding Vb as 

V B = {((x N \ A ,x A ),i) G S A : ((x N \ A , y A ), i) G V B for some y A G Z l f}. 

From the definition, it can be seen that {Vb, B G M a } is a partition of Sa- For y G Sa, define a 
function f{y) as 



T {y) 



T B if y G Vb for some B G M A such that B ^ A, 
1 otherwise, 



and a time sequence {i n } as to = 0, t n+ i = t n + T(Y tn ), n > 0. The embedded Markov chain 

is defined as Y n = Y^ n , n > 0. By the assumption of induction, we have, for B G M A such that 

B ^ A and for y G Vb, 

±E(xl 1 -Xl \Yl = y)=g l (B,y)n £ p B (C)a,(C). (4.9) 

For y G >U, gi(A,y) is represented as 

») = ^E{X^ Ta - X lfi \Y = y) 

1 Ta / 

A n=l 

+ l(t n _i < T A < i n ) {X hTA - Xj n _ 1 ) \Y\ = y 
= MB) + <h, (4-10) 

BeAf A 

where l(-) is an indicator function and, for B G M A , 

W5)4E E ^ (*Jn " *J„-l) I = V') P[K-l =y'\Yl = y) 

A n=l y / e y s 

and 

rp 

n=l 
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Because of the skip-free property of the chain and the assumption of r(y) = 1 for y G Va, we 
obtain 

M*) < ^ E E - A„-i I = V') p (Yn-i = y'\Y = y) 

A n=l y 'eV A 

^ Y~ A E P (^«-i e Va I y- = y), (4.11) 

?i=i 

and this implies that 0i(^4) ~ for a sufficiently large T4 because the induced Markov chain C A 
is not positive recurrent and the set {(x N \ A ,i) G 5 A : ((a?^" 4 , y^ 4 ), i) G Va for some y G is 
finite. We also have 

2 < maXB ^'^ TB , (4.12) 
Ta 

and this implies that 02 ~ for a sufficiently large T^. Since t n = i n _i + r(i ; "|j_ 1 ), we have, for 
B G M A , B ^ A, 

n=l j/'eVs 

x = y', t n _i + r(^Li) < ^ I Yl = y) 

T A 

= ^E E T B gi(B,y')P(Yl_ 1 = y',i n <T A \Yl = y), 

A n=l y/ e y s 

and this implies that 

<q A (B)e, (4.13) 



h(B)-q A (B) PB(C) ai (C) 



where 



</a(P) = ^E p (*^-i G < ^ | = y). 

A n=l 

Since r(y') = 1 for y' G Va, we define q A (A) as 

g A (A) = ^E P i^l G Va, t„ < T A | = y) . 

A 71=1 

Then, we have 



E <m(b) =^-E E EU{Y^\<yl_ x zV B )11h<T A ) 
^(E f &l-i) !(*n < ?a) I = y) , 



*0 
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and this implies 

1 - max ^ < Va(B) < 1. (4.14) 

On the other hand, qA(A) satisfies 
1 Ta 

q A {A)<—Y,P{Yn-i^VA\Y = y), (4.15) 

n=l 

and this implies that qA(A) ~ for a sufficiently large Ta because of the same reason as that why 
4>i(A) « for a sufficiently large Ta- Therefore, we obtain, for a sufficiently large Ta, 

0<1- max ^-q A (A)< £ q A (B) < 1, (4.16) 
A B£j\f ,B^A 

and from this, we see that vector (qA(B), B G J\f A ,B ^ A) is substochastic. For C € A^, let 
Pa(C) be defined as 

Pa(C)= Yl qa(b) Pb (c), 

B£Af A ,B^A 

then vector (pa(C), C G A/"^ 1 ) is also substochastic. As a result, from expression (|4.10j) . we obtain, 
for a sufficiently large Ta, 
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(A,y)- PA(C) m (C) 



c&N A 



< Y, 4>i(B)-q A (B) Y Pb(C)cli(C) 
B£M A ,B^A 



+ \MA)\ + \<h\ < e, (4-17) 



where Ka can be taken arbitrarily largely; This completes the proof. □ 
4.3 Main results 

Let U = (uj) = (ui j) be a d x d symmetric matrix, where Uj is the jth. column composing U, and 
define a set of d x d positive definite matrices, U, as 

U = |[7 = (uj) : U is positive definite and (a(A),Uj) < for all A G M p and all j G a|, 

where (2:1,2:2} is the inner product of vectors 2:1,2:2 G M rf and a(A) is the mean increment vector 
evaluated by the stationary distribution of the induced Markov chain C A which is positive recurrent. 
The following theorem provides a sufficient condition on which the (i-dimensional RRW-WBP is 
positive recurrent. 

Theorem 4.1. The d- dimensional RRW with a background process is positive recurrent if IA 7^ 0. 



11 



Proof. Let U = (v,j) = (uij) be a d x d positive definite matrix and introduce the following 
quadratic function as a test function: 

f(y) = f(x, i) = X - (x, Ux),y= (x, i) G S. (4.18) 

In order to prove the theorem, it suffices to show that there exists a positive definite matrix U 
satisfying the condition of Corollary 13.11 Let {Y n } be the embedded Markov chain of {^ n } 
defined in Subsection 14.1.31 We have X n +i = X n + Y^k=l™ £t«+fc' wriere £ k = X k — X k -i, and 
the difference between f(Y n+ i) and f(Y n ) satisfies 

, /r(Y tn ) r(Y tn ) \ /r(Y tn ) v 

f(Y n+1 ) - f(Y n ) = - ( tt n+k ,u £ €*»+*) + ( E ^*») 

\ k=l k=l I \ k=l I 

/r(Y tn ) V 

<co(r(yO,^) + ( E Zt n+ k,UX n V (4.19) 

where 

co(r(^J,?7) = ir(^J 2 max (sc, £/y) < oo 

2 x,y£{-l,0,l} d 

and we use the assumption that the RRW-WBP is skip free in all directions. Hence we obtain, for 
y = (x,i) G 5, 

E(f(Y n+1 ) - f(Y n ) \Y n = y)< c (r(y), U) + eU^ £ tn+k , Ux\ Y n = y) 

^ k=i ' 
= c (r(y), U) + r(y) (g(y), Ux). (4.20) 

Let A be a nonempty subset of N. For y = (x,i) G Va, we have xi < m.ax. B&J ^A B ^A^B for 
I £ N\A, and thus 

E(f(Y n+1 )-f(Y n )\Y n = y) 

<c (T A ,U) + T A V |<y(y),«,)| max K B + T A V(y(y), «j) x h (4.21) 

where we use the fact that (g(y),Ux) = Y2i=i(9(y)i u i) x i- The first and second terms on the 
right hand side of inequality (|4.2ip are bounded and do not depend on the value of Ka- On the 
other hand, we have x\ > Ka for all I G A. Thus, if (g(y),ui) < for all I € A, there exists a 
sufficiently large Ka such that E(f (Y n+ i) — f(Y n ) \ Y n = y) < — e for some e > 0. Therefore, by 
Corollary 13. 1\ in order to prove the theorem, it suffice to show that there exists a set of parameters 
{(T A ,K A ) : A C N, N 0} such that, for all A C N, A ^ 0, and for all / G ,4 and y G Va, we have 
(g(y), U[) < 0; it is obvious that V@ is finite and E(f(Y n+ i) \ Y n = y) < oo for all y G V0. 

Let £/ satisfy the condition of the theorem. That is, for all A G Af p and for all I G A, we have 
(a(.A),Uj) < 0. By Proposition 14. 1\ there exists a set of parameters {(Ta,Ka) '■ A C N, A ^ 0} 
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such that, for A C N, A / 0, and for y G Va and I G A 



(g{y),ui)< Yl PA(B){a(B),ui) + J2 9i(y)~ E PA(B) ai (B) 



< p A (B) (a(B), Ul ) + s'J2\ u ul ( 4 - 22 ) 



where e' > can be taken arbitrarily small. Thus, we obtain (g(y),ui) < 0. Since Ka can be 
taken arbitrarily largely in Proposition 14. 1\ this completes the proof. □ 

Next we consider a condition for the RRW-WBP to be transient. For A C N such that ^4^0, 
let M A and N A be index sets of induced Markov chains, defined as 

M A = \J JV« , Af A =Af A nM p , 
leA 

and let Wa be a set of d-dimensional vectors, defined as 

W A ={» = (w A , w n \ A ) eR d :w A > A , w N \ A < N \ A , 
and (a(B),w) > for all B G A/"/}, 

where = (0^,0^) is a vector of 0's with a suitable dimension. A sufficient condition on which 
the (i-dimensional RRW-WBP is transient is given by the following theorem. 

Theorem 4.2. The d- dimensional RRW with a background process, {Yn} = {(X n , J n )}, is tran- 
sient ifWA for some Ac N, A^%. 

Proof. Suppose that there exists A C N, A ^ 0, satisfying Wa ^ and let w = (w A ,w N \ A ) be 
an element of Wa- Define the following linear function as a test function: 

f(y) = f{x,i) = (x,w), y = (x,i) G S. 

Since the front process of is skip- free in all directions, we have 

|/(y n+ i) - f(Y n )\ = \(X n+1 - X n ,w)\ < ]T \wt\, (4.23) 

ieN 

and this implies that if \f(y') — f{y)\ > Szejv 1^1 then P{Y\ = y' \ Yq = y) = 0. Thus, we have 
b = ^2i eN \wi\ satisfying condition (iii) of Proposition 13.21 For a positive integer c, define a subset 
of S, A, as 

A = {y = (x, i) G S : f{y) = (x, w) > c}. 

Since w A > A and w N \ A < 0^, we have, for y = (x,i) G A, (x A ,w A ) > c. Thus, we can set c 
so large that Vb H A = for all B C N \ A, and this implies that 

AcS\ |J V B = (J V B - 

BcN\A BeM A 
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Hence, for all y G A, there exists B G M A such that y G V# and, by Proposition 14.11 we obtain, 
for the y, 

E(f{Y n+1 ) - f(Y n ) | Y n = y) = E({X n+1 - X n ,w) | Y n = y) 

= r{y) (g(y),w) 

*T B Y, PB(C)(a(C),w)>0, 
cert* 

where we use the fact that Mp C because of B n A ^ 0. This leads to condition (ii) of 
Proposition 13.21 and complets the proof. □ 

Remark 4.3. In the proof of the theorem, A is a half space of S separated by the hyperplane 
(x,w) = c, where w is a normal vector of the hyperplane. Hence, the condition of the theorem 
means that the expected time-averaged increment vector of any point y G A, g{y), does not cross 
the hyperplane. This well works for two-dimensional RRWs-WBP, but does not for a certain class 
of three-dimensional RRW-WBP, as mentioned in the next section. 

5 Low dimensional cases 

In this section, applying Theorems 14.11 and 14.21 we obtain specific conditions on which the d- 
dimensional RRW with a background process is positive recurrent and on which it is transient, 
when d = 2 and 3. Since a one-dimensional RRW with a background process is an ordinary 
discrete-time quasi-birth-and-death process (QBD process), a necessary and sufficient condition 
on which the QBD process is positive recurrent is well known (see, for example, Neuts [11] and 
Latouche and Rawaswami [7] ) ; We can derive the same result by Theorems 14.11 and 14.21 but it is 
straightforward and we omit it in the paper. 

5.1 Two dimensional case 

Consider the case of d = 2. Let {l^n} = {(X n , J n )} be a two-dimensional RRW with a background 
process, which is denoted by C. We assume that C is irreducible and that all the induced Markov 
chains generated from C are also irreducible. We have N = {1,2} and, for x = (x\,X2) € 

if x\ = and X2 = 0, 
if x\ > and x<i = 0, 
if x\ = and x<i > 0, 
otherwise. 

The state space S is given by 

S = ({0} x {0} x S ) U ({0} x N x u(Nx {0} x S {2} ) u(ff xNx S N ) . 

For A, B C N and z\, Z2 G {—1,0, 1}, we define a matrix of transition probabilities, P Zl ',z 2 , as 



(p(x) 
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where pf' B is given in expression (12. lh . We assume P^'zo to be a zero matrix for impossible 



at,{i} 



transitions; for example, we have P 1 [ 
of them with respect to z\ and/or z 2 by 



0. We abbreviate P^]t 2 to P£ Z2 and denote the sum 



P 



A,B 



A,B 



V p 

z\ £{-1,0,1} 



P. 



A,B 



P. 



A,B 



z 2 e{-i,o,i} 



P. 



A,B 



A,B 



/ j Z1,Z2 

*i,*ae{-i,o,i} 



The induced Markov chain £^ is an irreducible finite Markov chain, whose transition probability 
matrix is given by P^. Hence C N is positive recurrent and we denote by iv N its stationary 
distribution. The mean increment vector a(N) = (ai(N),i = 1,2) is given as 

ai(N) = <.( - P^ + P£) 1, a 2 (N) = <( - Pj% + p£) 1, 

where 1 is a column vector of l's with a suitable dimension. The induced Markov chains and 

£{2} 

are irreducible QBD processes whose transition probability matrices, denoted by P"W and 
p{ 2 \ respectively, are given in block form as follows: 



p{i} 



^ ■ 1 I 1 p.Y pN 



p, 



*,-l 



Pi 

* 

>N 

*,-l 



*,0 



pAT pTV 
^,0 ^,1 



V 



\ 



/ 



p{2} 



P 



7V,{2} 
1,* 



pN 



Pi 



N 



P 



N 



1.* 



\ 



\ 



J 



By Theorem 7.2.3 of Latouche and Rawaswami [7], for I € {1,2}, is positive recurrent if and 
only if a,3-i(N) < 0; By Theorem 6.2.1 of Latouche and Rawaswami [7], if is positive recurrent, 
its stationary distribution irW is given as, in matrix geometric form, 



7T 



{1} 



TV 



{'} 



4 } (RW) k -\ k > 2, 



where R^ is the rate matrix of . Hence, the mean increment vectors a({l}) = (aj({l}), % = 1, 2) 
and a({2}) = (aj({2}),i = 1,2) are given by 



ai({l}) 



7T 



{1}/ p{l} pW,JV , p {l} , p {l},iV\ 

-1,0 -1,1 ' ^1,0 ^1,1 I L 



+ ( - p5« - p5 1)0 - p5 m + p^ 1} + Pfo + PiTi) i 

+ 7 r«(/-P«)- 1 (-P_V + P 1 ^)l, 
a 2 ({l})=0, ai ({2})=0, 
« 2 ({2}) = 



I 2 ), 



P 



{2} 



,{2},JV 



{2} 



0,-1 



pl^/i 1 ' _i_ TDX^S , pi- 

M.,-1 M),l -'l, 



,{2},A^ 



, _{2}/ pJV,i^| pA? p JV , i p» i plV\i 

+ n l { ~ -1,-1 ~~ "0,-1 ~~ ^1,-1 "i -1,1 ' 0,1 ' 1,1/ 



>JV.{2} 



+ 7T 



{2} ( 



-1 , 



Classification of two-dimensional RRWs with a back ground process is given by the following 
theorem. 

Theorem 5.1. (CI) When ai(N) < and a 2 (N) < 0, the 2D-RRW-WBP {Y n } is 

(a) positive recurrent if ai({l}) < and a 2 ({2}) < 0; 
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(b) transient if either ai({l}) > or a2({2}) > 0. 
(C2) When a x (N) > and a 2 (N) < 0, the 2D-RRW-WBP {Y n } is 

(a) positive recurrent if ai({l}) < 0; 

(b) transient i/ai({l}) > 0. 

(C3) When ai(N) < and a 2 (N) > 0, the 2D-RRW-WBP {Y n } is 

(a) positive recurrent if a2({2}) < 0; 

(b) transient if a 2 ({2}) > 0. 

(C4) When ai(N) > and a 2 (N) > 0, the 2D-RRW-WBP {Y n } is transient. 

Proof. In (CI), we have J\f p = {{1}, {2}, N}. First we prove (Cl)-(a). In order for a square matrix 
U = (u\,u 2 ) = (uij,i, j = 1, 2) to be positive definite, it suffices that u\\ > and det U > 0, where 
we assume u\ 2 = u 2 \. Hence, if we find a solution to the following system of inequalities, then the 
solution is an element of IA in Theorem 14, It 

(a({l», ui) = ai({l}) un < 0, (o({2», u 2 ) = a 2 ({2}) u 22 < 0, 

{a(N), ui) = ai(iV) un + a 2 (N) u X2 < 0, (a(N), u 2 ) = ai(N) u 12 + a 2 (N) u 22 < 0, 

un > 0, u n u 22 - u\ 2 > 0. 

It is an easy task; For example, that U\i = u 22 = 1 and u\ 2 = 1/2 is such a solution. Thus, IA is 
not empty and, by Theorem 14. 11 the 2D-RRW-WBP is positive recurrent. Next we prove (Cl)-(b). 
Consider the case of oi({l}) > 0. We have A/W = {{1},N} and A/" p {1} = {{l},iV}. Hence, if we 
find a solution w = (wi,w 2 ) to the following system of inequalities, then the solution is an element 
of W{i} in Theorem 14.21 

(o({1}),«j) = ai({l})wi > 0, (a(N),w) = a 1 (N)wi+a 2 (N)w 2 >0, w 1 >0,w 2 <0. 

It is such a solution that wi = 1 and w 2 = —ai(N)/a 2 (N) — 5 < for some 5 > 0. Thus, W{\} is 
not empty and, by Theorem 14.21 the 2D-RRW-WBP is transient. The same result is analogously 
obtained in the case of a 2 ({2}) > 0. 

In (C2), we have M p = {{1}, N}. First we prove (C2)-(a). If we find a solution to the following 
system of inequalities, then the solution is an element of IA in Theorem I4.lt 

(a({l», ui) = ai({l}) un < 0, (a(N), Ul ) = ai(JV) u n + a 2 (N) u l2 < 0, 
(o(iV), u 2 ) = ai(iV) U12 + a 2 (iV) u 22 < 0, mh > 0, unu 2 2 - u\ 2 > 0. 

This is equivalent to 

ai(N) a 2 (N) 2 

< r-pr u n < u\ 2 < 7T7V U22, u n > 0, U22 > 0, u n u 22 > u 12 , 

a 2 (N) ai(N) 

and taking u 22 to be sufficiently large, we can obtain a solution of the system of inequalities. Thus, 
IA is not empty and, by Theorem l4.ll the 2D-RRW-WBP is positive recurrent. Next we prove (C2)- 
(b). We have = {{1},^} and A/" p {1} = {{l},iV}. Hence, if we find a solution w = (wi,w 2 ) to 
the following system of inequalities, then the solution is an element of W{i} in Theorem 14. 2\ 

(a({l}),u>) = ai({l}) 101 > 0, (a(N),w) = a 1 (N)w 1 +a 2 (N)w 2 >0, w 1 >0,w 2 <0. 
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It is such a solution that w\ = 1 and w 2 = — 1. Thus, W^} is not empty and, by Theorem 14.21 the 
2D-RRW-WBP is transient. (C3) is symmetric to (C2), thus the proof is analogous. 

In (C4), we have N p = {N} and N,^ = {N}. Hence, if we find a solution w = (101,102) to the 
following system of inequalities, then the solution is an element of Wn in Theorem 14.21 

(a(N),w) = oi(iV)wi + a 2 (N)w 2 > 0, wi > 0, w 2 > 0. 

It is such a solution that W\ = w 2 = 1. Thus, Wat is not empty and, by Theorem 14.21 the 
2D-RRW-WBP is transient. □ 

Remark 5.1. By Theorem 7.2.3 of Latouche and Rawaswami JW, if a\{N) = and a 2 (N) < 0, 
then C^ 1 } is positive recurrent and is not positive recurrent. Thus, we obtain the same result 
as (C2) in Theorem \4-l\ Analogously, if a\(N) < and a 2 (N) = 0, then we obtain the same 
result as (G3) in Theorem if (ai(N) > and a 2 (N) = 0) or (ai(N) = and a 2 (N) > 0) 
then the 2D-RRW-WBP is transient. However, in the case where ai(iV) = and a 2 (N) = 0, the 
2D-RRW-WBP cannot be classified by our theorems. 

5.2 Three dimensional case 

Consider the case of d = 3. In this case, we have iV = {1, 2, 3} and there are seven induced Markov 
chains: C is a finite Markov chain, £^' 2 \ C {2 ' 3} and £ {3 ' 1} are discrete-time QBD processes, and 
£« £W and are 2D-RRWs-WBP. Hence we can calculate the mean increment vectors a(N), 
a({l,2}), a({2,3}) and a({3, 1}) by standard methods, and classify the induced Markov chains 
£{ 2 } anc i £{3} by Theorem 15.11 However, as far as we know, there are no general methods to 
obtain the stationary distribution of a 2D-RRW-WBP, and hence it is difficult to obtain the mean 
increment vectors a({l}), a({2}) and a({3}) when the corresponding induced Markov chains are 
positive recurrent. Therefore, in the rest of the subsection, supposing that the signs of a({l}), 
a({2}) and a({3}) are known, we classify 3D-RRWs-WBP. 

The classification can be done in a manner similar to that used for the 2D-RRW-WBP, but in this 
case there are many combinations of the signs of the mean increments. Hence, we summarize only 
typical cases on Tables[T]to[H where, for example, "a({l, 2}) = (H — 0)" means that ai({l, 2}) > 0, 
a2({l, 2}) < and as({l,2}) = 0; "NA" means that the corresponding induced Markov chain is 
transient, blank means anything, and "positive" ("transient") means that the 3D-RRW-WBP, 
denoted by £, is positive recurrent (resp. transient). Of course, the tables does not contain all the 
cases, but every case is shown on some table or symmetric to someone on some tables; for example, 

the case where a(N) = ( ), o({l,2}) = ( 0), a({2,3}) = (0 + -), a({3, 1}) = (-0-), 

a({l}) = (— 0), a({2}) = (0 — 0) and a({3}) = NA is not shown on any table but it is symmetric 
to Case Cl-2-1 on Table [TJ We prove only Cases Cl-1-1 and Cl-1-2; Other cases except for Cl-7-1 
are analogously proved. 

Proof of Cl-1-1 and Cl-1-2. First we consider Cl-1-1. In this case, we have 

Af p = {{1}, {2}, {3}, {1, 2}, {2, 3}, {3, 1},N}. 

In order for a matrix U = (ux, 112,11,3) = (uij,i,j = 1,2,3) to be positive definite, it suffices that 
un > 0, unu 22 — u\ 2 > an d det U > 0, where we assume Uij = Uji for any i, j. Hence, if we find 
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a solution to the following system of inequalities, then the solution is an element of U in Theorem 

ED 



o({l}),ui) = ai({l})«n < 0, (o({2}),« a ) = a 2 ({2})u 2 2 < 0, 

a({3}),u 3 ) = a 3 ({3})u 33 < 0, 

a({l, 2}), ui) = oi({l, 2}) un + a 2 ({l, 2}) « 12 < 0, 

a({l, 2}), u 2 ) = ai({l, 2}) u 12 + a 2 ({l, 2}) u 22 < 0, 

a({2, 3}), u 2 ) = a 2 ({2, 3}) u 22 + a 3 ({2, 3}) u 23 < 0, 

a({2, 3}), u 3 ) = a 2 ({2, 3}) u 23 + a 3 ({2, 3}) u 33 < 0, 

a({3, 1}), u 3 ) = a 3 ({3, 1}) u 33 + oi({3, 1}) u 13 < 0, 

a({3, 1}), ui) = a 3 ({3, 1}) u 13 + ai({3, 1}) u n < 0, 

[a(N), Ul ) = a 1 (N)u 11 + a 2 (N)u 12 + a 3 (N)u 13 < 0, 

a(N),u 2 ) = a 1 {N)u 12 + a 2 {N)u 22 + a 3 {N)u 23 < 0, 

;o(iV),« 3 ) =a 1 {N)u 13 + a 2 (N)u 23 + a 3 (N)u 33 < 0, 

> 0, tinn 22 - uj 2 > 0, 
unu 22 -u 33 + 2-ui 2 n 23 ui 3 - unul 3 - u 22 uj 3 - u 33 u\ 2 > 0. 



For a 5 > 0, we set = u 22 = u 33 = 5 and ui 2 = u 23 = u% 3 = 1. Then, since all the mean 
increments are negative, the inequalities above except for ones on the bottom two lines holds; 
The fist inequality on the second line from the bottom obviously holds and, from the remaining 
inequalities, we obtain 



5 2 - 1 > 0, 5 3 + 2 - 35 > 0. 

For a sufficiently large 5, these two inequalities hold. Hence, IA is not empty and, by Theorem 14. 1\ 
the 3D-RRW-WBP is positive recurrent. 

Next we consider CI- 1-2. We have = Afj 1} = {{1}, {1, 2}, {3, 1}, N}. Thus, if we find a 

solution w = (wi,w 2 ,w 3 ) to the following system of inequalities, then the solution is an element of 
W{i} in Theorem 14.21 

(a({l}),w) = a 1 ({l})w 1 >0, 
(o({l, 2}), w) = oi({l, 2}) wi + a 2 ({l, 2}) w 2 > 0, 
(a({3,l}),u>> =a 1 ({3,l})«>i +a 3 ({3,l})w 3 > 0, 
(a(N),w) = ai(N) w a + a 2 (N) w 2 + a 3 (N) w 3 > 0, 
wi > 0, w 2 < 0, iu 3 < 0. 

For a 5 > 0, we set w\ = 5 and w 2 = w 3 = —1, then the inequalities are reduced to 
a 2 ({l,2}) a 3 ({3,l}) a 2 (N) + a 3 (N) 



< 6 < min 



oi({l,2})' ai({3,l})' ai(iV) 



and (5 satisfying this inequality exists. Therefore, W{i} is not empty and, by Theorem 14.21 the 
3D-RRW-WBP is transient. ' ' □ 
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C 1-7-1 on Table [T] is a unique case. In relation to semimartingale reflecting Brownian motions 
(SRBMs), Cl-7-1 corresponds to the case where a SRBM has spiral fluid paths (see, for example, 
Kharroubi et al. [6j and Bramson et al. [2]). The following theorem provides, for Cl-7-1, sufficient 
conditions on which the 3D-RRW-WBP is positive recurrent and on which it is transient; For 3D- 
RRWs without a background process, a similar result was obtained in a different way in Fayolle et al. 
[5]. For the case where the 3D-RRW-WBP is positive recurrent, the theorem is proved by Theorem 
14.11 On the other hand, for the case where the 3D-RRW-WBP is transient, it cannot directly be 
proved by Theorem 14.21 and some tricks are needed to prove it; Since the proof is complicate and 
lengthy, we put it in Appendix lAl 

Theorem 5.2. (a) Assume the 3D-RRW-WBP satisfies a(N) < and 

ai({l,2}) < 0, a 2 ({l,2}) > 0, a 2 ({2,3}) < 0, a 3 ({2,3}) > 0, 
a 3 ({3,l}) <0, oi({3,l}) >0. 



Then it is positive recurrent if 



a 2 ({l,2}) 



ax ({1,2}) 



a 3 ({2,3}) 



« 2 ({2,3}) 



and it is transient if 



« 2 ({1,2}) 



ai({1.2}) 



a 3 ({2,3}) 



a 2 ({2,3}) 



ai({3,l}) 



«3«3,1}) 



ax ({3,1}) 



as({3,l}) 



< 1, 



> 1. 



(5.1) 



(5.2) 



(b) Cl-7-1: Assume the 3D-RRW-WBP satisfies a(N) < and 

ai({l,2}) > 0, a 2 ({l,2}) < 0, a 2 ({2,3}) > 0, a 3 ({2,3}) < 0, 
a 3 ({3,l}) >0, oi({3,l}) <0. 



Then it is positive recurrent if 



« 2 ({1,2}) 



a 2 ({2,3}) 



«s({2,3}) 



as({3,l}) 



and it is transient if 



oi({l,2}) 



« 2 ({1,2}) 



a 2 ({2,3}) 



« 3 ({2,3}) 



oi({3,l}) 
a 3 ({3,l}) 



a a ({3,l}) 



< 1, 



> 1. 



(5.3) 



(5.4) 



Proof of the case where the 3D-RRW-WBP is positive recurrent. We prove only part (a); part (b) 
is analogously proved. We have 



AT P = {{1,2},{2,3},{3,1},JV}. 
Hence, for a symmetric matrix U = (u\, 112,113) 



u 



hJli 



if we find a solution to the following 
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system of inequalities, then the solution is an element of U in Theorem I4.lt 

(o({l, 2}), ui) = oi({l, 2}) un + a 2 ({l, 2}) u X2 < 0, 
(a({l, 2}), u 2 ) = ai({l, 2}) u 12 + a 2 ({l, 2}) n 22 < 0, 
(a({2, 3}), u 2 ) = a 2 ({2, 3}) u 22 + a 3 ({2, 3}) n 23 < 0, 
<a({2, 3}), « 3 > = ^({2, 3}) u 23 + a 3 ({2, 3}) n 33 < 0, 
(a({3, 1}), tig) = as({3, 1}) "33 + a x ({3, 1}) n 13 < 0, 
(a({3, 1}), «i) = a 3 ({3, 1}) ui 3 + ai({3, 1}) n n < 0, 
(a(iV), i/!) = ai(iV) nn + a 2 (iV) u 12 + a 3 (iV) ui 3 < 0, 
(a(N),u 2 ) = a 1 (N)u 12 + a 2 (N)u 22 + a 3 (N)u 23 < 0, 
(a(jV),u 3 ) = ai(iV)m 3 + a 2 (iV)n 23 + a 3 (7V)n 33 < 0, 
nn > 0, nnn 22 - u\ 2 > 0, 

"H"22"33 + 2ni 2 n 23 n 13 - u n ul 3 - u 22 u\ 3 - n 33 nf 2 > 0. 

Assume that > for all i,j and define ri 2 , r 23 and r 3 i as 

Q2({1,2» fl 3 ({2,3}) Qi({3,l» . n 

ri2 = m oiN > °> r 23 = /r» Qn > 0, r 3i = > 0, 

Qi({l,2» a 2 ({2,3}) a 3 ({3, 1}) 

where, by the assumption, we have ?"i 2 r 23 r 3 i < 1. Then, the inequalities are reduced to 

(5.5) 

"n"22 - u\ 2 > 0, (5.6) 
"n"22"33 + 2ni 2 n 23 n i3 - nun| 3 - u 22 uj 3 - u 33 u\ 2 > 0. (5.7) 

For a 5 satisfying < 5 < nn, set 

"22 = rf 2 2 (nn - 5), n 33 = rfi(nn + 5). (5.8) 

Then, letting 5 be sufficiently small, we have 

n 22 nn - 5 „ . r 23 "33 , n2 «ii + 5 „ r 3i u u u u 

— = < !, — = ( r 12^23?-3l) 7 < 1, — = — ? < 1, 

r 12 nn nn r 23 n 22 nn — o r 31 n 33 nn + o 

and this implies that there exist m 2 , n 23 and ni 3 satisfying inequalities (|5.5j) . Let c be defined as 
c = unn 22 n 33 > and, for a sufficiently small e > 0, set 

"12 = (c-e)/w33, " 2 3 = ( c - e)/«H, "13 = (c-e)/n 22 , (5.9) 



ri 2 n 22 < ni 2 < r 12 x nn, r 23 n 33 < n 23 < r 23 n 22 , 
r 3 i nn < "13 < 7~3i "33, 



then these u\ 2 , n 2 3 and ni 3 satisfy inequalities (15. 5p and (|5.6p . Substituting them into inequality 
(15.71). we obtain 



fc-e) 3 

- --2c + 3e>0, (5.10) 

c 

and this holds for any e > since the left hand side of inequality ()5.10|) is increasing in e € (0, oo) 

and takes the value of zero at e = 0. Therefore, we see that the system of inequalities (15. 5p to 
(|5.7p has a solution. Thus, W is not empty and, by Theorem 14.11 the 3D-RRW-WBP is positive 

recurrent. □ 
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Table 1: Classification of 3D-RRWs-WBP: a(N) = ( 



")• 



Case 


o({l,2}) 


a({2,3}) 


a({3,l}) 


«({!}) 


a({2}) 


a({3}) 


c 


Cl-1-1 


(- - o) 


(0 - -) 


(-0-) 


(-0 0) 


(0 - 0) 


(00-) 


positive 


Cl-1-2 


(- - o) 


(0 - -) 


(-0-) 


(+0 0) 






transient 


Cl-1-3 


(- - o) 


(0 - -) 


(-0-) 




(0 + 0) 




transient 


Cl-1-4 


(- - o) 


(0 - -) 


(— -) 






(00+) 


transient 


Cl-2-1 


(+ - o) 


(0 - -) 


(-0-) 


(-0 0) 


NA 


(oo-) 


positive 


Cl-2-2 


(+ - o) 


(0 - -) 


(- -) 


(+0 0) 


NA 




transient 


Cl-2-3 


(+ - o) 


(0 - -) 


(— — ) 




NA 


(oo+) 


transient 


Cl-3-1 


(+ + o) 






NA 


NA 




transient 


C 1-4-1 


(+-o) 


(o + -) 


(-0-) 


(-0 0) 


NA 


NA 


positive 


C 1-4-2 


(+-o) 


(o + -) 


(-0-) 


(+0 0) 


NA 


NA 


transient 


Cl-5-1 


(+-o) 


(o-+) 


(-0-) 


(-0 0) 


NA 


(oo-) 


positive 


Cl-5-2 


(+-o) 


(o - +) 


(-0-) 


(+0 0) 


NA 




transient 


C 1-5-3 


(+-o) 


(o-+) 


(-0-) 




NA 


(oo+) 


transient 


Cl-6-1 


(+-o) 


(o + -) 


(+0-) 


(-00) 


NA 


NA 


positive 


C 1-6-1 


(+ - o) 


(o + -) 


(+0-) 


(+0 0) 


NA 


NA 


transient 


Cl-7-1 


(+ - o) 


(o + -) 


(-0+) 


NA 


NA 


NA 


(The. I5T2|) 




Table 2: 


Classification of 3D-RRWs-WBP 


: a(N) = 


(+- ") 




Case 


a({l,2}) 


a({2,3}) 


a({3,l}) 


«({!}) 


a({2}) 


a({3}) 


C 


C2-1-1 


(- - o) 


NA 


(-0-) 


(-0 0) 


(0 - 0) 


(0 0-) 


positive 


C2-1-2 


(- - o) 


NA 


(-0-) 


(+0 0) 






transient 


C2-1-3 


( — o) 


NA 


(-0-) 




(0 + 0) 




transient 


C2-1-4 


(- -0) 


NA 


(-0-) 






(0 0+) 


transient 


C2-2-1 


(+-o) 


NA 


(-0-) 


(-0 0) 


NA 


(oo-) 


positive 


C2-2-2 


(+-o) 


NA 


(-0-) 


(+0 0) 


NA 




transient 


C2-2-3 


(+-o) 


NA 


(-0-) 




NA 


(0 0+) 


transient 


C2-3-1 


(+ + o) 


NA 




NA 


NA 




transient 


C2-4-1 


(+-o) 


NA 


(+0-) 


(-0 0) 


NA 


NA 


positive 


C2-4-2 


(+-o) 


NA 


(+0-) 


(+0 0) 


NA 


NA 


transient 


C2-5-1 


(+-o) 


NA 


(-0+) 


NA 


NA 


(00-) 


positive 


C2-5-2 


(+-o) 


NA 


(-0+) 


NA 


NA 


(0 0+) 


transient 
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Table 3: Classification of 3D-RRWs-WBP: a(N) = (+ + -). 



Case 


a({l,2}) 


a({2,3}) 


a({3,l}) 


«({!}) 


a({2}) 


a({3}) 


C 


C3-1-1 


(--o) 


NA 


NA 


(-0 0) 


(o-o) 


NA 


positive 


C3-1-2 


(- - o) 


NA 


NA 


(+0 0) 




NA 


transient 


C3-1-3 


(- -0) 


NA 


NA 




(o + o) 


NA 


transient 


C3-2-1 


(+-o) 


NA 


NA 


(-00) 


NA 


NA 


positive 


C3-2-2 


(+-o) 


NA 


NA 


(+0 0) 


NA 


NA 


transient 


C3-3-1 


(+ + o) 


NA 


NA 


NA 


NA 


NA 


transient 





Table 4: 


Classification 


of 3D-RRWs-WBP: 


a(N) = 


(+ + +) 




Case 


a({l,2}) 


a({2,3}) 


o({3,l}) o({l}) 


a({2}) 


a({3}) 


C 


C4-1-1 


NA 


NA 


NA NA 


NA 


NA 


transient 



5.3 An example of 3D-RRW-WBP 

In order to understand a feature of the 3D-RRW-WBP, we consider a simple model of three queues 
with service interactions (see Fig. [2]). Each queue is modeled as an M/M/l queue with sever 
vacations and input rejections; When a queue becomes empty, the server of the queue takes an 
exponentially distributed vacation with the common mean of 1/6 and customers arriving at the 
queue during the vacation are rejected. After ending the vacation, the sever enters an idle state 
and waits for customers to arrive. We denote the queues by Qi, Q2 and Q3, respectively, and 
denote by A the common arrival rate and by /i the common service rate; We assume [i > A. When 
all the severs are idle or busy for service, the behavior of the queues are mutually independent. 
However, when some server takes a vacation, an interaction between the queues occurs as follows: 
For / G {1, 2, 3}, when the server of Qi takes a vacation, the server of Q(/ +1 ) mod 3 begins to suspend 
its service; When the server of Q/ ends the vacation and becomes idle, the server of Q( i+1 ) m od 3 
resumes its service. Here we consider positive recurrence of this model when the value of 5 varies. 

Let N be given as N = {1,2,3}. For I G N, let Xi(t) be the number of customers at Q/ at 
time t, and define X(t) as X(t) = (Xi(t), I G N). For I G N, let Ji{t) be the state of the server 
of Q; at time t, and define J(t) as J(t) = (Ji(t), I G N); "J;(i) = 2" means that the server is 
serving a customer, u Ji(t) = 1" the server is suspending a service, u Ji(t) = 0" the server is idle and 
u Jl(t) = —1" the server is on a vacation. Then, {Y(t)} = {(X(t), J(t))} becomes a continuous- 
time version of 3D-RRW-WBP, where the state subsets of the background process on the boundary 
faces except for are given by 

= {(2,2,2)}, = {(1,2,-1), (2,2,0)}, 

5 {2,3} = Xj 2)j (0> 2j 2)}> £{3,1} = {(2> _ x> 1}> (2) 0> 2)}j 

S« = {(1,-1,-1), (2,-1,0), (1,0,-1), (2,0,0)}, 
S& = {(-1,1,-1), (-1,1,0), (0,2,-1), (0,2,0)}, 
S< 3 > = {(-1,-1,1), (-1,0,2), (0,-1,1), (0,0,2)}. 
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Let S be the state space of {Y(t)} and, for 2/1,2/2 £ <S, 2/i 7^ y 2 , let 9(2/1,2/2) be the transition 
rate from y l to y 2 . Applying the uniformization technique to {Y(t)}, we obtain a discrete-time 
3D-RRW-WBP, denoted by {l^n,} = {(X n ,J n )}, where the transition probabilities of {l^n} are 
given by, for a sufficiently large v > and for 2/1,2/2 € 5, 



P{Y n+l =y 2 \Y n = y t ) 



<?(l7i, 2/2)/^ 2/i^2/2> 

1 -E W 'ss,w' ? 4» 1 9(vi>i' / )/ I/ yi = yz- 



iri this model, the state space of the induced Markov chain £ w includes only one state, and the 
mean increments with respect to C are given by a\(N) = a 2 (N) = a 3 (iV) = — (/i — X)/u < 0. 
Thus, the induced Markov chains C^- 1 ' 2 ^, C^ 2 ' 3 ^ and C^ 3 ' 1 ^ are positive recurrent and they have the 
same stationary distribution. For example, the stationary distribution of C^ 1,2 ^ = {(A^3, n , <^3,n)}, 
^{1,2} = k = _l ? 0, 1, ...), is given by 

P-i = \ 1 ~ P xn , p k = p k {5/\)p^ k>0, 
1 - p + b/X 

where p_i = P(X 3 = 0, J 3 = -1), p = P{X 3 = 0, J 3 = 0), p fc = P(X 3 = k,J 3 = 2), fe > 1, and 
p = X/p; in deriving this stationary distribution, we use the fact that Q3 is independent of Qi 
and Q2 in the situation where the induced Markov chain C^ 1 ' 2 ^ is considered. From the stationary 
distribution, we obtain the following mean increments: 

01 ({1, 2}) = a 2 ({2, 3}) = a 3 ({3, 1}) = -((1 - p_i)/j - X)/u, 
o 2 ({1,2})=o 3 ({2,3})=oi({3 j 1}) = -(m-A)/i/<0 j 
a 3 ({l,2}) =ai({2,3}) =a 2 ({3,l}) =0. 

First we assume 5 > Xp. Then we have ai({l, 2}) = a2({2, 3}) = a 3 ({3, 1}) < 0, and the induced 
Markov chains £W, and are positive recurrent. The state of the induced Markov chain 
C^ 1 } consists of the number of customers in Q2, that in Q 3 and the states of the servers in Q2 and 
Q 3 . Since service progress in Q 3 may be suspended due to the vacation of the server in Q2, the 
stationary probability that the sever is on a vacation in Q 3 is less than or equal to that in isolated 
Q 3 , i.e., 

P(the sever is on a vacation in Q 3 ) < 
Thus, by the assumption that 5 > Xp, we obtain 

ai({l}) = — Ml — P(the sever in Q 3 is on a vacation))/^ — X^/v 

<-((l-p_ 1 )M-A)/ I /<0, 

and this implies ai({l}) = a2({2}) = a 3 ({3}) < 0. Therefore, Case Cl-1-1 holds and the 3D- 
RRW-WBP is positive recurrent. Next we assume 5 < Xp. Then we have a± ({1, 2}) = a2({2, 3}) = 
a 3 ({3, 1}) > and the induced Markov chains C^- 1 *, and are transient. This case, 

therefore, corresponds to C 1-7-1. Thus, if Xp > 5 > A (p — 1/2), then we have 



ai({l,2}) 



« 2 ({1,2}) 



« 2 ({2,3}) 



«3«2,3}) 



«3«3,1}) 



ai({3,l}) 



A - (1 -^V<L 
p — X 



and, by Theorem l5.2t the 3D-RRW-WBP is positive recurrent; if A (p— 1/2) > 5, then it is transient. 
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Q 2 



Q 3 






I I I 



Figure 2: A three-queue model with service interactions. 



6 Conclusions 

We have provided sufficient conditions on which a ci-dimensional skip-free RRW with a background 
process is positive recurrent and on which it is transient, and applying the results, we have classified 
two and three-dimensional RRWs with a background process. However, our results cannot be 
applied to the case where the model is probably null recurrent; for example, the case where the 
mean increment vector a(N) evaluated by the stationary distribution of the induced Markov chain 
C N is equal to a zero vector, i.e., a(N) = 0. To understand recurrence and transience of the model 
in such a case is an interesting problem and we let it be a further study. 
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A Proof of Theorem 15.21 

The case where the 3D-RRW-WBP is transient. We prove only part (a); part (b) is analogously 
proved. First we define a test function. Set c 2 and c 3 so that they satisfy 

a 2 ({l,2}) c 3 a 3 ({2,3}) 1 ai({3,l» 

< C2 < m oTT ' < — < < — < A.l 

-01 ({1,2}) c 2 -a 2 ({2,3}) c 3 -a 3 ({3, 1}) 

It is possible because of inequality ()5.2|) . Let Co and c\ be sufficiently large positive numbers and 
define vectors iO/ 1)2 }, ^ {2,3} an d ^{3,1} as 

If : , ,| - — ( !-■ — • — -CQ^ , W{ 2 ,3} = ^ f 1 - co, ^, ^ 







1 


1 


ci 


c 2 


C3 


ci 




1 

C2 


- c 



1 

W {3,1} 

ci \ c 2 c 3/ 

In terms of these vectors, we define, for A £ {{1, 2}, {2, 3}, {3, 1}}, a function f A as 

f A (x) = (x,w A ), x = (x 1 ,x 2 ,x 3 ) £ M 3 . 
From inequalities (|A.1|) . we obtain 



/ {1)2} (a({l, 2})) = - f ai ({l, 2}) + a 2 ({l, 2})- \ > 0, (A.2) 
/ {2 , 3} (a({2, 3})) = - ( a 2 ({2, 3})- + a 3 ({2, 3})-) > 0, (A.3) 

ci V c 2 c 3 y 

/ {3 , 1} (a({3, 1})) = - fa 3 ({3, 1})- + ai ({3, 1})) > 0. (A.4) 
a \ c 3 / 
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Furthermore, since all the elements of a(N) are negative and cq is positive and sufficiently large, 
we have 

/{i,2}(a(X» = - (ai(N) + a 2 (N)- + a 3 (N)- - a 3 (N) c ) > 0, (A.5) 
x s ci V c 2 c 3 ) 

f {2 ,3 } (a(N)) = - L(iV) + a 2 (N)- + a 3 (iV)- - ai(iV) cq) > 0, (A.6) 

Cl V c 2 C 3 ) 

f {3tl} (a(N)) = -( ai (N) + a 2 (N)-+a 3 (N)--a 2 (N)co) > 0, (A.7) 
ci V c 2 c 3 y 

where iV = {1,2,3}. We define a positive number eo as 

e ° = „ rr ^ r „ min{,/U(a(A)), /A(a(iV))} > 0, 

Ae{{l,2},{2,3},{3,l}} 

which we use after. In order to apply Proposition 13. 2\ we define a test function / as, for y = 
(x,i) G S, 

f(y) = /(«M) = max{/ {12} (s), /{2,3}0), /{3,1}0*0} , 
and a subset A of S as 

-4={yG<S:/(y)>l}. 

The plane f{i, 2 }(x) = 1 intersects the xi-axis at (ci,0, 0), the X2-axis at (0, ciC2,0) and the x 3 -axis 
at (0, 0, — c ^^_ l ); the plane f{ 2 , 3 }{x) = 1 intersects the xi-axis at (— ^^j,0, 0), the X2-axis at 
(0, cic 2 ,0) and the x 3 -axis at (0, 0,cic 3 ); the plane f{ 3 ,i}(x) = 1 intersects the xi-axis at (ci,0, 0), 
the X2-axis at (0, — c ^^_ 1 ,0) and the a; 3 -axis at (0, 0, cic 3 ). Therefore, since we assumed that Co and 
ci are positive and sufficiently large, it can be seen from the definition of the partition {Va ■ A C N} 
that we have A C S \ V%. It is obvious that A ^ and A c ^ 0. 

Next we consider, for each y G S, which function is assigned to f{y). Define a function c(x) 
as, for x = (xi,x 2 ,x 3 ) G M 3 , 

{{1,2}, x\ > x 3 , x 2 > x 3 , 
{2,3}, x 2 >x 1 ,x 3 >x 1 , 
{3, 1}, x 3 > x 2 , xi > x 2 . 

Since we have 

/{2,3}0) - f{l >2 }(x) = —(X 3 - Xi), f {3jl} (x) - f {2 - 3 }(x) = —(Xi - X 2 ), 

C\ Ci 

/{l,2}O0 - f{3,l}( x ) = ^(^2 - ^3), 

f(y) is represented as, for y = (x,i) G S, 

f(y) = f(x,i) = f a{x) (x). 

Let y = (x,i) be an element of V{i j2 }j where x = (x\,x 2 ,x 3 ) G Z^. From the definition of V{i ; 2}, 
we see that x±,x 2 > -^{1,2} > Kn and x 3 < Kn- Thus, we have x\ > x 3 and x 2 > x 3 , and this 
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leads to cr(x) = {1,2}. Analogous results also hold for V{ 2 ,3} and V{ 31 }. As a result, we obtain, 
for AG {{1,2}, {2, 3}, {3,1}}, 

f(y) = f(x, i) = f A (x) if y = (x, i) G V A . (A.8) 

Now we come to the phase to evaluate the conditional expectation of the difference between 
f(Y n+ i) and f(Y n ), where {^"n} = {(X n , Jn)} is the embedded Markov chain defined in Subsec- 
tion HTL3J Let 4>(y) be defined as 

<P(y) = E(f(Y n+l ) - f(Y n ) \Y n = y),ye S, 

which is represented as 

M = E {f. {Xl )^i) " f*(X o) (Xo) I Y = y). 

Since {X n } is skip free in all directions, it is obvios that if \f(y\) — f(yo)\ > b then we have 
P(Yi = y 1 \Y = y Q ) = 0, where 

, 2 ( , 1 1 
b = — 1 + — + — + c 

Cl V c 2 C 3 

Thus, in order to prove the theorem by Proposition 13.21 it suffices to show that there exists a 
positive number e such that (f>(y) > e for all y G A. To this end, we divide A into three parts: 

A n Vjv, A n (V {lj2} u V {2 , 3 } u V {3 ,i } ) and A n (V {1} u V {2} u V {3} ) . 

(a) Assume y = (x,i) G Ad Vn- Since the induced Markov chain C N is positive recurrent, we 
obtain, by Proposition 14.11 

' , Y = y 



(y)>E(f a{x) (j^{X n -X n ^)\ 

= T N L {x) {g (y)) « IV / CT(a!) (a(JV)) > e 



where ff(y) is given in Subsection 14.21 and we use the fact that / <7 (jf 1 )(-^l) > fa(x )( -^i)- This 
implies that, for any small positive number 5 and for a sufficiently large T/v, we have 

Hv) > T N {eo -S)>0. (A.9) 

(b) For A G {{1, 2}, {2, 3}, {3, 1}}, assume y = (x, i) G A(1Va- Since the induced Markov chain 
C A is positive recurrent, we obtain, by Proposition 14. 1\ 

^(y)>B^^(X n -X n _i)j |Yo = y) = T A f A (g(y)) nT A f A (a(A)) >T A e . 

This implies that, for any small positive number <5 and for a sufficiently large T^, we have 

0(y) > T A (e - 5) > 0. (A.10) 

(c) Assume y = (ajji) G .A Pi Vm. Since the induced Markov chain C^y is transient, we use 
the same technique as that used in the proof of Proposition 14.11 Let {(X n , J n )} be the expanded 
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Markov chain and {Y^} = {{X.\, Jj)} the embedded Markov chain generated from £{i}; For 

the construction of {Y n }, especially its embedded points {i n }, see the proof of Proposition 14.11 
For the expanded Markov chain, we modify the functions / and a and denote them by / and a, 
respectively; / and a are given by, for y = (x, i) £ <S{i}, 

r {1,2}, yev {1>2} , 

f(y) = U(x)(x), a{x)=l {3,1}, !/eV (3il ), 

[ a(x), otherwise, 

where S^y is the state space of C{iy, V{i,2} arm ^{3,1} are * ne SUDse ts of S{iy obtained by expanding 
Vji^} and V^i}, respectively. Then, by Remark 14.21 we have 



= E{f(Y T{1} ) - f(Y ) | Y = y)) 

= E(i(i n < t {1} ) (f(Yl) - /crU)) 



n=l 



+ l(t n _i < T {1} < i n ) {f(Y T{1} ) - /(*-!_!)) | Vj = 1/ 



n=l " 1 

+ < T {1} < i„) /.^t^lrp, - Xl_j) \y\ = Vj 

= T {i}( E <h(B) + <h), (A.ii) 



where A/" {1} = {{1}, {1, 2}, {3, 1}, N} and, for B € M {1} 



i(b) = e e ^f 1 ^ ^ r {i}) - *U) i ^i-i 



{!} n =l y'={ x ',i')eV B 



y 



xP{Yl-i = y'\Yl = y), 



= Y~ E ^(U^n-l < < *») 4 ( xt_ (*T {1} " Xl_,) \Yl=V 

n=l 



We have 



l, k , ){ x Tm - xU\ < A + 1 + 1 + c \ , (A . 12) 

and this implies that </>2 ~ for a sufficiently large Tm. Furthermore, we have 

1 / 1 1 \ 1 T{1} 

|<M{1})I <- l + - + - + co — E P ( :P «- 1 E % I ^0 = V), (A-13) 
c i V c 2 c 3 J T {1} ^ 
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and this implies that 0i({l}) ~ for a sufficiently large I'm because the induced Markov chain 
£{1} is transient. For B € \ {{f }}, we have 



f T{1> 

h{B) = —Y, E T * f^'Mv')) Piyl-x = V', in < T {1} \Yl = y) 

I 1 } n =l y'=(x',i')eV B 



11 E E f Hx 'MB))P(Yi-i=y'J n <T {1} \Yl = y) 



Ti 



{1} 



=1 y'=(x',i')eV B 



>e q {1} {B), (A.14) 

where 

T T<1} 

q {1} (B) = -^E P{Yi-i G Vs, in < T {1} I n = y) . 

n=l 

Therefore, for any small positive number 5 and for a sufficiently large we have 

^y)>T {1} (s (q {1} ({l,2})+q {1} ({3,l})+q {iy (N))-S) > 0, (A.15) 

where we use the fact that g{!}({l,2}) + q^y({3, 1}) + q{i}(N) > (see expression (|4.16p in the 
proof of Proposition 14, f p . Analogously we obtain, for y G inV{2} and for a sufficiently large T{ 2 }, 



^y)>T m (s (q {2} ({l,2})+q {2} ({2,3}) + q {2} (N))-Sj > 0, (A.16) 
and for y £ A n V{3} and for a sufficiently large 7{3}, 

Hy)>T {3} (eo{q {3} ({2,3})+q {3} ({3,2})+q {3} (N)) - s) > 0. (A.17) 



From the results of (a), (b) and (c), it can be seen that there exists a positive number e such 
that, for any y £ A, <f>{y) > e, and this completes the proof. □ 
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